1. Technical Field
The present invention relates to vehicle modeling and re-identification.
2. Discussion of the Related Art
In general, object shapes are complex. Shape representation has consistently been a challenge to computer vision applications. However, for the same class of objects, e.g., human faces, the variability is considerably smaller than the ensemble of all object shapes. Thus, it is possible to statistically model the shape of a certain class. A successful approach in this area is the so-called active shape model (ASM) [T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. Active shape models—their training and application. Computer Vision and Image Understanding, 61(1):38-59, January 1995]. The essence of an ASM is that an object class can be modeled by an average shape plus a small set of allowed variations. We now illustrate this point more precisely.
For instance, an object shape can be represented by a set of three-dimensional (3D) points xn, n=1, 2, . . . , N. These points can be a point cloud representation for a surface of an object, vertices of a triangular mesh, or points corresponding to edges defined by geometric discontinuities, high curvature regions and those defined by sharp surface reflectance changes. We can stack the points together and form a length 3N dimensional vector xx=[x1T,x2T, . . . , xNT]T,  (1)where T means transpose of a vector. Suppose that we have K samples from an object class, e.g., K different faces, we can have K such shape vectors x(k), k=1, 2, . . . , K. We assume that the same elements in two different vectors correspond. For example, the first element of two shape vectors x(1) and x(2) are both X coordinates of two person's nose tips. An ASM is thus represented by a mean shape m and a set of M variability vectors vm, m=1, 2, . . . , M. The mean shape m and the variability vectors are learned from the set of training samples and are fixed once learned. The variability of the shapes comes from a control vector λ. Once determined, the shape of an object instance can be approximated byx≈f(m,V,λ),  (2)where V={v1, v2, . . . , vM} is the set of variability vectors and f is the shape assembly function.
A simple case of the ASM is the linear ASM, where the shape assembly is simply a linear combination of the variability vectors and the mean shape,x=m+V˜λ,  (3)where by abusing symbols we write the shape variability matrix as V=[v1, v2, . . . , vM]. A convenient and effective way to build an ASM is by principle component analysis (PCA). That is,
                    m        =                              1            K                    ⁢                                    ∑                              k                =                1                            K                        ⁢                          x                              (                k                )                                                                        (        4        )            and we take vm as the first M eigenvectors (corresponding to the largest M eigenvalues) of the (semi-)definite symmetric matrix
                              ∑                      k            =            1                    K                ⁢                              (                                          v                k                            -              m                        )                    ⁢                                    (                                                v                  k                                -                m                            )                        T                                              (        5        )            To extract meaningful ASMs we usually require that K>>M.
Note that a prior distribution on the control parameters λ can be learned from the samples as well. A Gaussian prior is usually assumed on the control parameters,λ˜N(μλ,Σλ).  (6)For the linear ASM, μλ is a zero vector and Σλ is diagonal.
In the physical world, there is no guarantee that shape variability of an object class can be captured accurately by a linear ASM. However, besides convenience there are a few important reasons for adopting PCA ASM. First, a PCA ASM captures the majority of the shape variability. Given enough variability vectors vm, the approximation can be very accurate. Second, it is a convenient way of navigating through the space of infinitely many object shapes in a class. A linear model has the advantage that the optimal shape control vectors can be globally found by solving a least squares optimization problem.
ASMs have seen a great deal of success in computer vision applications, such as medical image processing [X. S. Zhou, D. Comaniciu, and A. Gupta. An information fusion framework for robust shape tracking. PAMI, 27(1): 115-129, 2005] and face tracking and modeling [V. Blanz and T. Vetter. Face recognition based on fitting a 3d morphable model. PAMI, 25(9):1063-1074, 2003 and J. Xiao, S. Baker, I. Matthews, and T. Kanade. Real-time combined 2D+3D active appearance models. In CVPR, 2004]. However, we have not seen applications of ASM to true 3D objects such as the class of consumer vehicles. There are a few reasons for this. First, an ASM is relatively easy to build from mostly 2D objects such as a slice of a CT/ultrasonic scan or mostly frontal views of human faces. The cases for vehicles are quite different. A vehicle can potentially be viewed from any angle and the vehicle shapes can look drastically different from two different angles. Second, aligning a vehicle ASM with an image observation is considerably more difficult. Unlike the face model where a low dimensional appearance model can be extracted from a set of training images, the appearance of the vehicles varies unpredictably as a function of surface material type, color, as well as the environment radiance map. In the case of face tracking and modeling, an ASM is usually combined with a PCA appearance model to form an active appearance model (AAM). Model/image registration is relatively easy by minimizing the sum of squared difference (SSD) between an observed image and a synthesized image. However, this is not the case for cars. Third, the shapes of vehicles are quite different from one another. Just imagine the differences among a pickup truck, a passenger sedan and a mini-van. The shape variability poses huge challenges for both shape representation and matching models with images.
Accordingly, there exists a need for a technique of applying an ASM to true 3D objects such as the class of consumer vehicles.